Characteristics of EEG power spectra involved in the proficiency of motor learning.

Neuromodulation techniques for modulating brain activity can affect performance in a variety of behaviors. Techniques including transcranial alternating current stimulation and random noise stimulation can modulate neural oscillations. However, the intervention effect of neuromodulation approaches on motor learning is poor, partly because the electroencephalography (EEG) power spectra associated with the motor learning process has not yet been fully elucidated. Therefore, we investigated the characteristics of EEG power spectra in the process of motor learning in 15 right-handed healthy participants (5 females; mean age = 22.8 ± 3.0 years). The motor task was a ball-rotation task in which participants rotated two balls in the palm of their left hand. Participants performed a pre-test, the motor learning tasks, and a post-test. In the motor learning tasks, twenty 60 s trials were performed in the clockwise (CW) direction. Before and after the motor learning tasks, CW and counterclockwise (CCW; control condition) tasks were performed for 60 s each as pre- and post-tests. Therefore, CW direction was set as a motor learning task, while CCW was a test-only control task. EEG was recorded during the tests and tasks, and the power spectra in the alpha, beta, and gamma oscillations were calculated and compared between pre- and post-tests. The results showed that in the CW post-test, the power of the gamma band in the left parietal areas and the right frontal area was significantly higher than in the pre-test. In the CCW, there was no significant difference in each band at each area between the pre- and post-tests. Our findings reveal the characteristics of the EEG spectra related to the motor learning process. These results may help to establish more effective neuromodulation approaches to modifying neural oscillations in motor learning, including in rehabilitation fields.

Evaluation of expert skills in refinery patrol inspection: visual attention and head positioning behavior.

We aimed to clarify expert skills in refinery patrol inspection using data collected through a virtual reality experimental system. As body positioning and postural changes are relevant factors during refinery patrol inspection tasks, we measured and analyzed both visual attention and head positioning behavior among experts and "knowledgeable novices" who were engaged in the engineering of the refinery but had less inspection experience. The participants performed a simulated inspection task, and the results showed that 1) expert inspectors could find more defects compared to knowledgeable novices, 2) visual attention behavior was similar between knowledgeable novices and experts, and 3) experts tended to position their heads at various heights and further from the inspection target to obtain visual information more effectively from the target compared to knowledgeable novices. This study presented the differences in head positioning behavior between expert and novice inspectors for the first time. These results suggest that to evaluate the skills used in inspecting relatively larger targets, both visual attention and head positioning behavior of the inspectors must be measured.

Path-Planning System for Radioisotope Identification Devices Using 4π Gamma Imaging Based on Random Forest Analysis.

We developed a path-planning system for radiation source identification devices using 4π gamma imaging. The estimated source location and activity were calculated by an integrated simulation model by using 4π gamma images at multiple measurement positions. Using these calculated values, a prediction model to estimate the probability of identification at the next measurement position was created by via random forest analysis. The path-planning system based on the prediction model was verified by integrated simulation and experiment for a 137Cs point source. The results showed that 137Cs point sources were identified using the few measurement positions suggested by the path-planning system.

Clarify Sit-to-Stand Muscle Synergy and Tension Changes in Subacute Stroke Rehabilitation by Musculoskeletal Modeling.

Post-stroke patients exhibit distinct muscle activation electromyography (EMG) features in sit-to-stand (STS) due to motor deficiency. Muscle activation amplitude, related to muscle tension and muscle synergy activation levels, is one of the defining EMG features that reflects post-stroke motor functioning and motor impairment. Although some qualitative findings are available, it is not clear if and how muscle activation amplitude-related biomechanical attributes may quantitatively reflect during subacute stroke rehabilitation. To better enable a longitudinal investigation into a patient's muscle activation changes during rehabilitation or an inter-subject comparison, EMG normalization is usually applied. However, current normalization methods using maximum voluntary contraction (MVC) or within-task peak/mean EMG may not be feasible when MVC cannot be obtained from stroke survivors due to motor paralysis and the subject of comparison is EMG amplitude. Here, focusing on the paretic side, we first propose a novel, joint torque-based normalization method that incorporates musculoskeletal modeling, forward dynamics simulation, and mathematical optimization. Next, upon method validation, we apply it to quantify changes in muscle tension and muscle synergy activation levels in STS motor control units for patients in subacute stroke rehabilitation. The novel method was validated against MVC-normalized EMG data from eight healthy participants, and it retained muscle activation amplitude differences for inter- and intra-subject comparisons. The proposed joint torque-based method was also compared with the common static optimization based on squared muscle activation and showed higher simulation accuracy overall. Serial STS measurements were conducted with four post-stroke patients during their subacute rehabilitation stay (137 ± 22 days) in the hospital. Quantitative results of patients suggest that maximum muscle tension and activation level of muscle synergy temporal patterns may reflect the effectiveness of subacute stroke rehabilitation. A quality comparison between muscle synergies computed with the conventional within-task peak/mean EMG normalization and our proposed method showed that the conventional was prone to activation amplitude overestimation and underestimation. The contributed method and findings help recapitulate and understand the post-stroke motor recovery process, which may facilitate developing more effective rehabilitation strategies for future stroke survivors.

The over-estimation of distance for self-voice versus other-voice.

Self-related stimuli are important cues for people to recognize themselves in the external world and hold a special status in our perceptual system. Self-voice plays an important role in daily social communication and is also a frequent input for self-identification. Although many studies have been conducted on the acoustic features of self-voice, no research has ever examined the spatial aspect, although the spatial perception of voice is important for humans. This study proposes a novel perspective for studying self-voice. We investigated people's distance perception of their own voice when the voice was heard from an external position. Participants heard their own voice from one of four speakers located either 90 or 180 cm from their sitting position, either immediately after uttering a short vowel (i.e., active session) or hearing the replay of their own pronunciation (i.e., replay session). They were then asked to indicate which speaker they heard the voice from. Their voices were either pitch-shifted by ± 4 semitones (i.e., other-voice condition) or unaltered (i.e., self-voice condition). The results of spatial judgment showed that self-voice from the closer speakers was misattributed to that from the speakers further away at a significantly higher proportion than other-voice. This phenomenon was also observed when the participants remained silent and heard prerecorded voices. Additional structural equation modeling using participants' schizotypal scores showed that the effect of self-voice on distance perception was significantly associated with the score of delusional thoughts (Peters Delusion Inventory) and distorted body image (Perceptual Aberration Scale) in the active speaking session but not in the replay session. The findings of this study provide important insights for understanding how people process self-related stimuli when there is a small distortion and how this may be linked to the risk of psychosis.

AdjustSense: Adaptive 3D Sensing System with Adjustable Spatio-Temporal Resolution and Measurement Range Using High-Speed Omnidirectional Camera and Direct Drive Motor.

Many types of 3D sensing devices are commercially available and were utilized in various technical fields. In most conventional systems with a 3D sensing device, the spatio-temporal resolution and the measurement range are constant during operation. Consequently, it is necessary to select an appropriate sensing system according to the measurement task. Moreover, such conventional systems have difficulties dealing with several measurement targets simultaneously due to the aforementioned constants. This issue can hardly be solved by integrating several individual sensing systems into one. Here, we propose a single 3D sensing system that adaptively adjusts the spatio-temporal resolution and the measurement range to switch between multiple measurement tasks. We named the proposed adaptive 3D sensing system "AdjustSense." In AdjustSense, as a means for the adaptive adjustment of the spatio-temporal resolution and measurement range, we aimed to achieve low-latency visual feedback for the adjustment by integrating not only a high-speed camera, which is a high-speed sensor, but also a direct drive motor, which is a high-speed actuator. This low-latency visual feedback can enable a large range of 3D sensing tasks simultaneously. We demonstrated the behavior of AdjustSense when the positions of the measured targets in the surroundings were changed. Furthermore, we quantitatively evaluated the spatio-temporal resolution and measurement range from the 3D points obtained. Through two experiments, we showed that AdjustSense could realize multiple measurement tasks: 360∘ 3D sensing, 3D sensing at a high spatial resolution around multiple targets, and local 3D sensing at a high spatio-temporal resolution around a single object.

Classification of Motor Impairments of Post-Stroke Patients Based on Force Applied to a Handrail.

Many patients suffer from declined motor abilities after a brain injury. To provide appropriate rehabilitation programs and encourage motor-impaired patients to participate further in rehabilitation, sufficient and easy evaluation methodologies are necessary. This study is focused on the sit-to-stand motion of post-stroke patients because it is an important daily activity. Our previous study utilized muscle synergies (synchronized muscle activation) to classify the degree of motor impairment in patients and proposed appropriate rehabilitation methodologies. However, in our previous study, the patient was required to attach electromyography sensors to his/her body; thus, it was difficult to evaluate motor ability in daily circumstances. Here, we developed a handrail-type sensor that can measure the force applied to it. Using temporal features of the force data, the relationship between the degree of motor impairment and temporal features was clarified, and a classification model was developed using a random forest model to determine the degree of motor impairment in hemiplegic patients. The results show that hemiplegic patients with severe motor impairments tend to apply greater force to the handrail and use the handrail for a longer period. It was also determined that patients with severe motor impairments did not move forward while standing up, but relied more on the handrail to pull their upper body upward as compared to patients with moderate impairments. Furthermore, based on the developed classification model, patients were successfully classified as having severe or moderate impairments. The developed classification model can also detect long-term patient recovery. The handrail-type sensor does not require additional sensors on the patient's body and provides an easy evaluation methodology.

Perception and control: individual difference in the sense of agency is associated with learnability in sensorimotor adaptation.

Adaptive motor learning refers to the ability to adjust to novel disturbances in the environment as a way of minimizing sensorimotor errors. It is known that such processes show large individual differences and are linked to multiple perceptual and cognitive processes. On the other hand, the sense of agency refers to the subjective feeling of control during voluntary motor control. Is the sense of agency just a by-product of the control outcome, or is it actually important for motor control and learning? To answer this question, this study takes an approach based on individual differences to investigate the relationship between the sense of agency and learnability in sensorimotor adaptation. Specifically, we use an adaptive motor learning task to measure individual differences in the efficiency of motor learning. Regarding the sense of agency, we measure the perceptual sensitivity of detecting an increase or a decrease in control when the actual level of control gradually increases or decreases, respectively. The results of structure equation modelling reveal a significant influence of perceptual sensitivity to increased control on motor learning efficiency. On the other hand, the link between perceptual sensitivity to decreased control and motor learning is nonsignificant. The results show that the sense of agency in detecting increased control is associated with the actual ability of sensorimotor adaptation: people who are more sensitive in detecting their control in the environment can also more quickly adjust their behaviors to novel disturbances to acquire better control, compared to people who have a less sensitive sense of agency. Finally, the results also reveal that the processes of increasing control and decreasing control may be partially independent.

Deceleration Assistance Mitigated the Trade-off Between Sense of Agency and Driving Performance.

Driving assistance technology has gained traction in recent years and is becoming more widely used in vehicles. However, drivers usually experience a reduced sense of agency when driving assistance is active even though automated assistance improves driving performance by reducing human error and ensuring quick reactions. The present study examined whether driving assistance can maintain human sense of agency during early deceleration in the face of collision risk, compared with manual deceleration. In the experimental task, participants decelerate their vehicle in a driving simulator to avoid collision with a vehicle that suddenly cut in front of them and decelerated. In the assisted condition, the system performed deceleration 100 ms after the cut-in. Participants were instructed to decelerate their vehicle and follow the vehicle that cut-in. This design ensured that the deceleration assistance applied a similar control to the vehicle as the drivers intended to, only faster and smoother. Participants rated their sense of agency and their driving performance. The results showed that drivers maintained their sense of agency and improved driving performance under driving assistance. The findings provided insights into designing driving assistance that can maintain drivers' sense of agency while improving future driving performance. It is important to establish a mode of joint-control in which the system shares the intention of human drivers and provides improved execution of control.

Image Captioning Using Motion-CNN with Object Detection.

Automatic image captioning has many important applications, such as the depiction of visual contents for visually impaired people or the indexing of images on the internet. Recently, deep learning-based image captioning models have been researched extensively. For caption generation, they learn the relation between image features and words included in the captions. However, image features might not be relevant for certain words such as verbs. Therefore, our earlier reported method included the use of motion features along with image features for generating captions including verbs. However, all the motion features were used. Since not all motion features contributed positively to the captioning process, unnecessary motion features decreased the captioning accuracy. As described herein, we use experiments with motion features for thorough analysis of the reasons for the decline in accuracy. We propose a novel, end-to-end trainable method for image caption generation that alleviates the decreased accuracy of caption generation. Our proposed model was evaluated using three datasets: MSR-VTT2016-Image, MSCOCO, and several copyright-free images. Results demonstrate that our proposed method improves caption generation performance.

Modified sensory feedback enhances the sense of agency during continuous body movements in virtual reality.

The sense of agency refers to the feeling of control over one's own actions, and through them, the external events. This study examined the effect of modified visual feedback on the sense of agency over one's body movements using virtual reality in healthy individuals whose motor control was disturbed. Participants moved a virtual object using their right hand to trace a trajectory (Experiment 1) or a leading target (Experiment 2). Their motor control was disturbed by a delay in visual feedback (Experiment 1) or a 1-kg weight attached to their wrist (Experiment 2). In the offset conditions, the virtual object was presented at the median point between the desired position and the participants' actual hand position. In both experiments, participants reported improved sense of agency in the offset condition compared to the aligned condition where the visual feedback reflected their actual body movements, despite their motion being less precise in the offset condition. The results show that sense of agency can be enhanced by modifying feedback to motor tasks according to the goal of the task, even when visual feedback is discrepant from the actual body movements. The present study sheds light on the possibility of artificially enhancing body agency to improve voluntary motor control.

Categorical Perception of Control.

The self is a distinct entity from the rest of the world, and actions and sensory feedback are our channels of interaction with the external world. This study examined how the sense of control influences people's perception of sensorimotor input under the framework of categorical perception. Twenty human participants (18 males, two females) took part in both experiments. Experiment 1 showed that the sensitivity (d') of detecting a 20% change in control from no change was higher when the changes occurred at the control-category boundary than within each category. Experiment 2 showed that the control categories greatly affected early attention allocation, even when the judgment of control was unnecessary to the task. Taken together, these results showed that our perceptual and cognitive systems are highly sensitive to small changes in control that build up to a determinant change in the control category within a relatively narrow boundary zone between categories, compared with a continuous, gradual physical change in control.

Robust and Efficient Indoor Localization Using Sparse Semantic Information from a Spherical Camera.

Self-localization enables a system to navigate and interact with its environment. In this study, we propose a novel sparse semantic self-localization approach for robust and efficient indoor localization. "Sparse semantic" refers to the detection of sparsely distributed objects such as doors and windows. We use sparse semantic information to self-localize on a human-readable 2D annotated map in the sensor model. Thus, compared to previous works using point clouds or other dense and large data structures, our work uses a small amount of sparse semantic information, which efficiently reduces uncertainty in real-time localization. Unlike complex 3D constructions, the annotated map required by our method can be easily prepared by marking the approximate centers of the annotated objects on a 2D map. Our approach is robust to the partial obstruction of views and geometrical errors on the map. The localization is performed using low-cost lightweight sensors, an inertial measurement unit and a spherical camera. We conducted experiments to show the feasibility and robustness of our approach.

The Active Sensing of Control Difference.

In everyday life, people control objects in the world around them to varying degrees. The processes people actively use to establish their control, while interacting with an environment containing large ambiguity, remain unknown. This study examines how people explore their control over the environment and how they detect small differences in control among objects. In the experimental task, participants moved three dots on a screen and identified one dot over which the level of control is different from that of the other two. The results support a two-step behavior mechanism underlying the sensing of control difference: People first explore their overall control in the environment, and then the results of the initial exploration are used to selectively tune the direction (i.e., either more or less) of the detected control difference, ensuring efficient and rapid detection of the type of control difference that is potentially important for further action selections.

Acoustic Inspection of Concrete Structures Using Active Weak Supervision and Visual Information.

Concrete structures are featured heavily in most modern societies. In recent years, the need to inspect those structures has been a growing concern and the automation of inspection methods is highly demanded. Acoustic methods such as the hammering test are one of the most popular non-destructive testing methods for this task. In this paper, an approach to defect detection in concrete structures with active weak supervision and visual information is proposed. Based on audio and position information, pairs of samples are actively queried to a user on their similarity. Those are used to transform the feature space into a favorable one, in a weakly supervised fashion, for clustering defect and non-defect samples, reinforced by position information. Experiments conducted in both laboratory conditions and in field conditions proved the effectiveness of the proposed method.

The Sense of Agency in Driving Automation.

Driving automation has been developing rapidly during the latest decade. However, all current technologies of driving automation still require human drivers' monitoring and intervention. This means that during driving automation, the control by human driver and by the driving automation system are blended. In this case, if the human driver loses the sense of agency over the vehicle, he/she may not be able to actively engage in driving, and may excessively rely on the driving automation system. This review focuses on the subjective feeling of agency of the human driver over the vehicle in such situations. We address the possible measures of agency in driving automation, and discuss the insights from literatures on the sense of agency in joint control, robotics, automation, and driving assistance. We suggest that maintaining the sense of agency for human driver is important for ethical and safety reasons. We further propose a number of avenues for further research, which may help to better design an optimized driving automation considering human sense of agency.

Temporal Features of Muscle Synergies in Sit-to-Stand Motion Reflect the Motor Impairment of Post-Stroke Patients.

Sit-to-stand (STS) motion is an important daily activity, and many post-stroke patients have difficulty performing STS motion. Previous studies found that there are four muscle synergies (synchronized muscle activations) in the STS motion of healthy adults. However, for post-stroke patients, it is unclear whether muscle synergies change and which features primarily reflect motor impairment. Here, we use a machine learning method to demonstrate that temporal features in two muscle synergies that contribute to hip rising and balance maintenance motion reflect the motor impairment of post-stroke patients. Analyzing the muscle synergies of age-matched healthy elderly people ( n = 12 ) and post-stroke patients ( n = 33 ), we found that the same four muscle synergies could account for the muscle activity of post-stroke patients. Also, we were able to distinguish post-stroke patients from healthy people on the basis of the temporal features of these muscle synergies. Furthermore, these temporal features were found to correlate with motor impairment of post-stroke patients. We conclude that post-stroke patients can still utilize the same number of muscle synergies as healthy people, but the temporal structure of muscle synergies changes as a result of motor impairment. This could lead to a new rehabilitation strategy for post-stroke patients that focuses on activation timing of muscle synergies.

Organization of functional modularity in sitting balance response and gait performance after stroke.

BACKGROUND: Recovery of postural adjustment, especially when seated, is important for performing activities of daily living after stroke. However, conventional clinical measures provide little insight into a common strategy for dynamic sitting balance and gait. We aimed to evaluate functional re-organization of posture and ambulatory performance after stroke. METHODS: The subjects of the study included 5 healthy men and 21 post-stroke patients. The spatiotemporal modular organization of ground reaction forces during a balance task in which the leg on the non-affected side was lifted off the ground while seated was quantified by using complex principal component analysis. FINDINGS: A 3% decrease in the temporal strength of the primary module in post-stroke patients was an independent predictor of gait performance in the hospital setting with high sensitivity and specificity. Tuning of the temporal strength was accompanied by the recovery of sitting and ambulation. INTERPRETATION: Our findings suggest that evaluation of the modular characteristics of ground reaction forces during a sitting balance task allows us to predict recovery and functional adaptation through daily physical rehabilitation.

Improvement of Sense of Agency During Upper-Limb Movement for Motor Rehabilitation Using Virtual Reality.

Sense of agency refers to the feeling of controlling one's own body. Many patients surviving from a stroke lose the sense of agency over their body. This is due to impairments in both motor control and sensory brain functions. As a result of this lack in the sense of agency, stroke patients tend to lose the intention of moving the paralyzed limb, which results in further deterioration of brain functions and worsening muscles and joints. The present study proposes a motor rehabilitation system using virtual reality to improve the sense of agency during upper-limb movement which is required for various daily life activities such as eating meals and operating devices. Specifically, participants were instructed to move their hand to track a moving target ball in a virtual reality environment, while the position of their real hand was measured via a motion capture system. Participants were shown another ball presenting the position of their hand in virtual reality. We tested the proposed system with healthy participants, of which the motor control was disturbed by a 1-kg weight attached on the wrist. Participants reported their sense of agency after each trial. The results showed that the sense of agency was enhanced by the proposed intervention. Our results pointed out a potentially useful method to improve the sense of agency during body movements using modified visual feedback, which may contribute to the development of rehabilitation for stroke patients.

Disposable Stiffness Sensor for Endoscopic Examination.

Since direct manual palpation is not possible in minimally invasive procedures, there is an active field of applied research which aims to retrieve the human sense of touch and feedback tissue properties through artificial tactile feedback. This paper presents an innovative stiffness sensor to be embedded at the tip of a commercial endoscopic camera. The sensor structure is based on multiple cantilever beams, which act as springs with different stiffness when indented into soft tissue. Geometric features mounted on the beams are tracked during physical contact. Movements of thecantilevers result in shape variations of the features in the camera images. The feature size is then segmented and related to the force exerted into the contact location. As beams of different elasticity are integrated, it is possible to estimate the stiffness properties of the soft tissue by employing only visual information. In this paper, Finite Element Analysis (FEA) was implemented to simulate and estimate how contact forces will affect the material and design of the prototype. A calibration device has been developed and used to validate the outcome of the FEA simulations. An experimental test showed the ability of the proposed mechanism to compute the stiffness of a soft phantom.